On hidden Markov chains and finite stochastic systems

On hidden Markov chains and finite stochastic systems

In this paper we study various properties of finite stochastic systems or hidden Markov chains as they are alternatively called. We discuss their construction following different approaches and we also derive recursive filtering formulas for the different systems that we consider. The key tool is a simple lemma on conditional expectations.

Taylor Expansion for the Entropy Rate of Hidden Markov Chains

We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...

Lectures on finite Markov chains

Hidden hybrid Markov/semi-Markov chains

Models that combine Markovian states with implicit geometric state occupancy distributions and semi-Markovian states with explicit state occupancy distributions, are investigated. This type of model retains the flexibility of hidden semi-Markov chains for the modeling of short or medium size homogeneous zones along sequences but also enables the modeling of long zones with Markovian states. The...

Stochastic Reachability: From Markov Chains to Stochastic Hybrid Systems

Stochastic hybrid systems represent established classes of realistic models of hybrid dynamics subject to random perturbations, autonomous uncontrollable transitions, nondeterminism or uncertainty. Stochastic reachability analysis is a key factor in the verification and deployment of stochastic hybrid systems. In this paper, we tackle the stochastic reachability problem from a purely probabilis...